1 |
Le modèle Transformer: un « couteau suisse » pour le traitement automatique des langues
|
|
|
|
In: Techniques de l'Ingenieur ; https://hal.archives-ouvertes.fr/hal-03619077 ; Techniques de l'Ingenieur, Techniques de l'ingénieur, 2022, ⟨10.51257/a-v1-in195⟩ ; https://www.techniques-ingenieur.fr/base-documentaire/innovation-th10/innovations-en-electronique-et-tic-42257210/transformer-des-reseaux-de-neurones-pour-le-traitement-automatique-des-langues-in195/ (2022)
|
|
BASE
|
|
Show details
|
|
3 |
Representation learning of natural language and its application to language understanding and generation
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Neural-based Knowledge Transfer in Natural Language Processing
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Impact of textual data augmentation on linguistic pattern extraction to improve the idiomaticity of extractive summaries
|
|
|
|
In: Lecture Notes in Computer Science ; https://hal.archives-ouvertes.fr/hal-03271380 ; Matteo Golfarelli; Robert Wrembel. Lecture Notes in Computer Science, Springer, In press, Lecture Notes in Computer Science (2021)
|
|
BASE
|
|
Show details
|
|
6 |
The Mapping of Deep Language Models on Brain Responses Primarily Depends on their Performance
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-03361439 ; 2021 (2021)
|
|
Abstract:
Recent deep networks like transformers not only excel in several language tasks, but their activations linearly map onto the human brain during language processing. Is this functional similarity caused by specific factors, such as the language abilities and the architecture of the algorithms? To address this issue, we analyze the brain responses to isolated sentences in a large cohort of 102 subjects, each recorded with both functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). We then compare the ability of 32,400 transformer embeddings to linearly map onto these brain responses. Finally, we evaluate how the architecture, training, and performance of the models independently account for this brain mapping. Our analyses reveal two main findings. First, the similarity between brain responses and the activations of language models primarily depends on their ability to predict words from the context. Second, this similarity allows us to decompose and precisely track the rise and maintenance of perceptual, lexical, and compositional representations within each cortical region. Overall, this study evidences a partial convergence of language transformers to brainlike solutions, and shows how this phenomenon helps unravel the brain bases of natural language processing.
|
|
Keyword:
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [SCCO.NEUR]Cognitive science/Neuroscience; Encoding; Functional Magnetic Resonance Imaging; Magneto-encephalography; Natural Language Processing
|
|
URL: https://hal.archives-ouvertes.fr/hal-03361439 https://hal.archives-ouvertes.fr/hal-03361439/file/Mous_mapping.pdf https://hal.archives-ouvertes.fr/hal-03361439/document
|
|
BASE
|
|
Hide details
|
|
7 |
A logistic regression model for predicting child language performance ; Un modèle de régression logistique pour la prédiction du développement langagier chez l'enfant
|
|
|
|
In: SIS 2021, 50th Annuale Conference of the Italian Statistical Society" ; https://hal.archives-ouvertes.fr/hal-03318721 ; SIS 2021, 50th Annuale Conference of the Italian Statistical Society", Jun 2021, Pise, Italy (2021)
|
|
BASE
|
|
Show details
|
|
8 |
Hierarchical-Task Reservoir for Online Semantic Analysis from Continuous Speech
|
|
|
|
In: ISSN: 2162-237X ; IEEE Transactions on Neural Networks and Learning Systems ; https://hal.inria.fr/hal-03031413 ; IEEE Transactions on Neural Networks and Learning Systems, IEEE, 2021, ⟨10.1109/TNNLS.2021.3095140⟩ ; https://ieeexplore.ieee.org/abstract/document/9548713/metrics#metrics (2021)
|
|
BASE
|
|
Show details
|
|
10 |
On Multi-domain Sentence Level Sentiment Analysis for Roman Urdu ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Multilingual Email Zoning - Segmenting Multilingual Email Text Into Zones
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
DiS-ReX: A Multilingual Dataset for Distantly Supervised Relation Extraction ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
DiS-ReX: A Multilingual Dataset for Distantly Supervised Relation Extraction ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Rule-based Morphological Inflection Improves Neural Terminology Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|